Multimodal Interaction Techniques for Joint Cooperation Between Humans and Cognitive Systems
نویسنده
چکیده
In this paper, we present an overview on recent advances in our work on multimodal human-machine communication between humans and technical cognitive systems in order to enable the solution of complex problems that require the joint cooperation of humans and intelligent systems to accomplish challenging tasks that can be only successfully handled if both interact in a cooperative manner. The technical system has cognitive capabilities that support an intelligent interaction with the human operator. An example for such cooperation is the joint assembly of a heavy part by a human assisted by a cognitive robot during the manufacturing process of a complex product. As will be explained, such challenges in joint cooperation occur often in intelligent manufacturing environments which provide a very rich application scenario for the successful combination of advanced multimodal human-machine communication techniques together with cognitive components for improved, cooperative interaction between technical systems and human operators.
منابع مشابه
Multimodal Interaction: Methods and Applications for Joint Cooperation between Humans and Cognitive Systems
In this paper, we present an overview on recent advances in our work on multimodal human-machine communication between humans and technical cognitive systems in order to enable the solution of complex problems that require the joint cooperation of humans and intelligent systems to accomplish challenging tasks that can be only successfully handled if both interact in a cooperative manner. The te...
متن کاملToward Multimodal Human-Robot Cooperation and Collaboration
Our multimodal interface integrates speech recognition, natural language understanding, spatial reasoning and human cognitive models for completing specific tasks and for perspective-taking in locative oriented tasks. With natural language and gestures, we believe human-robot interaction and communication is facilitated. Instead of concentrating on the various modalities of the interface, users...
متن کاملMultimodal Software for Affective Education: UI Evaluation
This paper focuses on evaluation techniques for multimodal software. We suggest how multimodal educational systems can help us and give a proposal for how to evaluate these systems. Our goal is to come up with human-computer interaction (HCI) evaluation techniques for multimodal systems to support affective education and adapt usability evaluation taking into consideration the cognitive walkthr...
متن کاملIntegrated Task Learning and Kinesthetic Teaching for Human-Robot Cooperation
We present an integrated framework that permits implicit task learning and kinesthetic teaching during the execution of robotic tasks in cooperation with humans. The proposed system combines physical human-robot interaction, attentional supervision, multimodal interaction to support robot teaching and incremental task learning. We describe the overall system architecture discussing a task learn...
متن کاملLearning joint multimodal behaviors for face-to-face interaction: performance & properties of statistical models
We evaluate here the ability of statistical models, namely Hidden Markov Models (HMMs) and Dynamic Bayesian Networks (DBNs), in capturing the interplay and coordination between multimodal behaviors of two individuals involved in a face-to-face interaction. We structure the intricate sensory-motor coupling of the joint multimodal scores by segmenting the whole interaction into so-called interact...
متن کامل